Repository navigation
fix(gateway): hold a request while its engine's lease drains - #340
Merged
Merged
Conversation
A draining engine still answers its probe, so the gateway proxied the next request to it. On aphotic at 12:45:22 on 2026-11-15, the runner read Strata idle, began to unload it, and Strata took the next agent turn in the two seconds it spent stopping and ended it with 0 tokens (finish=length). resolve no longer treats an engine as live while its lease is draining, parking or releasing, and ensure waits for that lease to settle before it probes. Active again, the request goes on to the engine. Gone, the engine stops answering and the gateway starts it anew, waiting for the card as any start does.
T-Crypt
added a commit
that referenced
this pull request
Nov 15, 2026
…ning Merge main into next: hold requests while an engine drains (#340)
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On aphotic at 12:45:20 on 2026-11-15, the drain read Strata idle between two agent turns and began to unload it. Strata took about 4 seconds to stop and kept answering meanwhile. The next turn came through the gateway at 12:45:22, and Strata ended it with 0 tokens and
finish=length. The client got a reply that looked like success and had no answer in it.The gateway decides an engine is live when its probe answers, and a stopping engine still answers. It already skips a parked engine for the same reason.
Change
fit.Leavingreports when an engine's lease isdraining,parkingorreleasing.resolveno longer counts that engine as live, so the request takes the start path, which streams keep-alive comments while it waits.ensurereads the lease every 250 ms until it settles. If the lease is active again (a drain the holder refused), the request goes on to the engine. If the lease is gone, the engine stops answering, and the gateway starts it anew, waiting for the card like any start.The same wait covers two other paths. A switch to an engine that is still draining now starts it once the drain ends. Before, the switch saw "already up" and did nothing. A request for a ComfyUI that is parking now resumes it once it is parked.
Requests sent straight to an engine's own port bypass the gateway and are not covered. Strata answering a request with 0 tokens while it stops is a Strata bug, filed upstream as Niko1221/Strata#6055.
Proof
TestGatewayHoldsARequestWhileItsEngineDrainsfails at the test commit (answered while the engine drained: 200). With the fix, it holds the request while the lease readsdraining, then forwards it once to the engine when the lease readsactive.go test ./..., the Windows build and vet, and golangci-lint 2.14.0.